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Abstract —This paper studies a multi-period demand re¬ 
sponse management problem in the smart grid where multiple 
utility companies compete among themselves. The user-utility 
interactions are modeled by a noncooperative game of a Stack¬ 
elberg type where the interactions among the utility companies 
are captured through a Nash equilibrium. It is shown that 
this game has a unique Stackelberg equilibrium at which the 
utility companies set prices to maximize their revenues (within 
a Nash game) while the users respond accordingly to maximize 
their utilities subject to their budget constraints. Closed-form 
expressions are provided for the corresponding strategies of 
the users and the utility companies. It is shown that the multi¬ 
period scheme, compared with the single-period case, provides 
more incentives for the users to participate in the game. A 
necessary and sufficient condition on the minimum budget 
needed for a user to participate is provided. 

I. Introduction 

Demand Side Management (DSM) is an essential compo¬ 
nent of the smart grid as it captures important aspects of the 
interactions between utility companies (UCs) and consumers, 
including residential, commercial, industrial users and ve¬ 
hicles [1]. These aspects can be technical, such as using 
advanced metering infrastructure to improve the reliability 
and efficiency of the grid [1], or social through agreements 
between consumers and energy providers [2]. The cate¬ 
gorization of DSM includes energy efficiency, time-of-use 
pricing, and demand response [3]. Energy efficiency is about 
the long-term solutions to improve the overall efficiency; 
one example is replacing the air conditioning system of 
a building with a more efficient one. Through time-of-use 
pricing, companies can agree with the users that different 
prices are to be charged at different periods; these prices 
are agreed on beforehand by both parties. Demand Response 
Management (DRM) is the response of consumers’ demands 
to price signals from the UCs [4]. DRM allows companies to 
manage the consumers’ demands, either directly or indirectly, 
through incentive-based programs [5]. 

Game theory is a powerful mathematical tool that can 
lead to effective multi-person decision making [6]. Due to 
the nature of the smart grid which features different entities 
with conflicting objectives, applying game-theoretic methods 
can improve their reliability and efficiency [1], [7]-[16]. 
For a comprehensive survey of game-theoretic methods for 
communications, DSM, and microgrid systems in the smart 
grid, we refer to [1]. 

There are several works where game-theoretic methods 
have been applied to DSM/DRM [9]-[16]. An autonomous 
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DSM through scheduling of appliances has been imple¬ 
mented within a noncooperative game framework in [9]. 
The participants in such a game are energy users who are 
connected to the same UC and the outcome of the game is the 
power consumption schedule of appliances that minimizes 
the overall energy cost. Plug-in hybrid electric vehicles can 
sometimes have a surplus of energy, and it can be more 
beneficial to sell energy quickly instead of holding onto 
it. This motivates the work of [10] in which an energy 
trading noncooperative game between vehicle groups and 
distribution grids has been studied. In [11], joint consumers 
discomfort and billing costs minimization within a repeated 
game framework has been shown to lead to an optimal DSM 
mechanism. 

In the smart grid, there are numerous interactions between 
UCs and the end-users because of the two-way communica¬ 
tion infrastructure [2]. In order to take this fact into account, 
utilizing a multi-level framework such as Stackelberg games 
would be very useful [12]-[15]. A multi-level game-theoretic 
framework has been developed for demand response man¬ 
agement, where consumers choose their optimal demands in 
response to prices announced by different UCs [12]. This 
Stackelberg game was shown to have a unique Stackelberg 
equilibrium at which UCs maximize their revenues and end- 
users maximize their payoff functions. In this framework, 
UCs were the leaders of the game and users were the 
followers. This framework, though it effectively captures 
user-utility interactions, is limited to the single-period case. 
In the smart grid, users are expected to be able to schedule 
their energy consumption, store or sell surplus energy, based 
on their self-interests. With energy scheduling and storage, 
users might have a flexibility about when to receive a 
certain amount of energy, particularly for shiftable appliances 
[9]. Some energy consumption can be scheduled and some 
cannot. For example, users might be flexible about doing the 
laundry, but not much flexibility is there for refrigerators. 

It is worth mentioning that multi-level games for DRM 
have been studied in a limited context in the literature. For 
example, a four-stage Stackelberg game has been studied 
where three stages are at the leader-level (the utility retailer), 
and the fourth stage is at the consumer level [13]. Retailers 
choose the amount of energy to procure, and the sources to 
produce it, in addition to deciding on the price. Consumers 
respond to these prices through demand selection. This game 
is also a single-period game and it does not take into 
account the competition between the UCs, but, it incorporates 
other aspects of the decision making at the company-level. 
Furthermore, a multi-period Stackelberg game with a single 


UC has been formulated with energy storage devices at the 
consumers-level [14]. Simulation results show that a Peak-to- 
Average Ratio minimization problem leads to similar deci¬ 
sions made within the framework of the game. Additionally, 
a noncooperative Stackelberg game between plug-in electric 
vehicle groups such as parking lots and the smart grid 
was formulated and a socially optimal equilibrium has been 
achieved [15]. A two-level game (a noncooperative game 
between multiple UCs and an evolutionary game for the users 
at the lower level) has been proposed in [16], but this game 
is also limited to the single-period case. 

There is a need to develop an analytical multi-period, 
multi-utility, and a multi-level framework. By introducing 
multi-period inter-temporal constraints, we could study a 
generalization of [12] to the multi-period case. Our work 
differs from the work in [ 12 ] at both the user-side and 
the company-side. At the user-side, we have an additional 
minimum energy constraint that needs to be satisfied across 
all periods, while at the company-side, we provide an alter¬ 
native computationally cheap closed-form solution for the 
prices. Having such a multi-period framework can make 
it possible to accommodate numerous extensions, such as, 
energy scheduling and storage, and Peak-to-Average ratio 
minimization. To recap, the multi-period scheme has been 
considered before in the literature, but most of the studies 
are either user-oriented or UC-oriented. Our goal here is to 
develop a reliable multi-period framework where numerous 
multi-level interactions are also taken into account. 

Accordingly, we formulate in this paper a Stackel¬ 
berg game for multi-period-multi-company demand response 
management. We derive solutions in closed-form and find 
precise expressions for the maximizing demands at the users’ 
level, and the revenue-maximizing prices for the UCs. We 
also prove the existence and uniqueness of the Stackelberg 
equilibrium. This work captures the competition between 
UCs and the budget limitations at the consumer-level. 

The remainder of this paper is organized as follows. 
The problem is formulated in Section HH The Stackelberg 
game along with its solutions in closed form are provided 
in Section [III] In Section |IV] some numerical results are 
provided. We conclude the paper in Section |V] with some 
remarks on future directions. 

II. Problem Formulation 

The system consists of > 1 UCs and A > 1 end-users. 
We consider a finite time horizon including T > I periods. 
Let ,^ = {1,2,..., A} be the set of UCs, ^ = {1,2,... ,A} 
be the set of end-users, and ^ = {1,2,... ,T} be the finite 
set of time slots. The unit of time can be hour, day, week, 
or month. Mathematically, it does not matter which unit the 
set .3^ represents. 

We model the interaction between UCs and end-users as 
a Stackelberg game. Thus, the DRM problem is formulated 
as two optimization problems, one is for the followers of 
the game and the other is for the leaders of the game. 
The followers of the game are the consumers, who can 
be residential, commercial, or industrial, and the leaders 


of the game are the UCs. On the followers (users) side, 
each user has a certain amount of energy demand to meet 
and we assume that the users have flexibility to schedule 
their consumption of energy. It is also assumed that users 
have limited budgets. The goal of each user is to maximize 
her utility without exceeding the budget while meeting the 
minimum energy demand. On the leaders (UCs) side, the 
goal of each UC is to maximize its revenue while its price 
setting is influenced by both the competition against other 
UCs and the behavior of the users. In the game considered 
here, the UCs announce their prices for each period to the 
users, and then the users respond accordingly by scheduling 
their demands. We discuss the underlying models for both 
sides separately below. 

A. User-Side 

Because of energy scheduling and storage, users may have 
some flexibility about when to receive a certain amount of 
energy. In our analysis, we are concerned about the total 
amount of shiftable energy. For non-shiftable energy, one 
can add some period-specific constraints. Additionally, each 
user has a budget constraint that she cannot exceed for the 
entire time horizon. 

The demand of user n G JV from UC k G JU at time t € ^ 
is denoted by and pk{t) denotes the price of energy 

announced by UC k at time t. Each end-user n has a budget 
of B„, and a minimum amount of energy to be met, denoted 
by E™”. The value of can be thought of as the total 
energy needed by user n during the considered total period 
(for example, one day or one week). User payoff functions 
increase by receiving more energy. The utility of user n is 
defined as 

Uusei,n — Yn ^ ^ lll(Cn T )) (^) 

keJK^teS 

where 7 ,, and (typically, = 1 ) are adjustment param¬ 
eters. The logarithmic function is well known to provide 
a good demand response [12], [17]. Users aim to maxi¬ 
mize their payoffs as much as possible while meeting this 
threshold of minimum amount of energy and not exceeding 
a certain budget. To be more precise, given B„ > 0, > 0, 

and pk{t) > 0 , the user-side optimization problem is defined 
as follows: 


maximize 

*in,k 

^user,« 


subject to 

keJ^te^ 

( 2 ) 


E E dn,kit) > Er 

kejff tesr 

(3) 


dn.k{t) > 0 , Mk^JU, 

(4) 


Note that there is no game played among the users. Each user 
responds to the price signals using only her local information. 
These price signals depend on all the demands selected 
by the users and hence users indirectly affect each other’s 
decisions. 


B. Company-Side 

Given the prices of other UCs and the power availability 
of UC k at period t, denoted by p_k and Gic{t) respectively, 
the total revenue for UC k is given by 

P—k) = ^ Pkil) ^ (5) 

The utility-side optimization problem is then described as 
follows: 

maximize Ugen,P-k) 

Pk 

subject to ^ dn^k{t) < Gk{t), V f G ( 6 ) 

Pkit)>0, \/te3r (7) 

The goal of each UC is to maximize its revenue and 
hence maximize its proht. Also, companies want to meet 
the consumers’ demands without exceeding the maximum 
power availability (overloading the system can cause contin¬ 
gencies). Additionally, because of the market competition, 
the prices announced by other companies also affect the 
determination of the price at company k. So, company k’s 
price selection is actually a response to what other UCs in the 
market have announced; this response is also constrained by 
the availability of power. Thus, there is a Nash game between 
utility companies. 

III. The Stackelberg Game 

In this section, we hrst solve the above optimization 
problems in closed-form. Then, we study the existence and 
uniqueness of Nash equilibrium at the UC-level and the 
Stackelberg equilibrium for the entire two-level game. 

A. Followers-Side Analysis 

Note that the user-side utility function is convex and the 
constraints are linear. We start by relaxing the minimum 
energy constraint (O and then find the necessary budget 
that makes the maximizing demands feasible. For each user 
n G , the associated Lagrange function is given as follows: 

Tu,ser,n — Yn ^ 

keJK'teS 

^n.l I ^ ^ P{^)k^n,k{^) j 

\keJkte^ J 

T ^ ^ ^ti,2{kd')dn,kit) 
kejf teS' 

where A„.,’s are the Lagrange multipliers. For optimality, by 
Krush-Kuhn-Tucker necessary conditions [18], 

= VfG,i7, vkGjr (8) 

odnM\t) 

^n.li Y. Y Pk^^)‘^n,k{t)-B,\ =0 (9) 

Xkejxrte^ J 

K.2{k,t)d„.kit) = Q, MkCJF ( 10 ) 

>0, Vf G ,5^, VkG,^ (11) 

The above conditions are also sufficient because of the 
convexity of the optimization problem [19]. In the sequel, we 
derive the solution in closed form by discussing two cases. 


1) Case 1: d„^k{t) > 0,k G JC,t G In this case, since 
■^n.i > 0 (constraint (|2|i is active [18]) and Xn.y.{kd) = 0 for 
d\\ t C SC and k G JC, there holds 


<^Lus( 


Yn 


ddfi j^{t^ dfi j^it^ 


- K,lPk{t) ^Q,y t G S", k€ JC 


From (fTSli and (|9ll, it is straightforward to verify that 


( 12 ) 


dn,kit) = -Cn,VtGSr,kGJC 


KTpkit) 


(13) 


which is a generalization of the single-period case considered 
in [12]. 


2) Case 2: at least one dn,k{t) is zero: We show that 
the expression (fTSi) also holds for d„ k(t) > 0, k G J(f, t G 
Without loss of generality, suppose that dn.\{i-) =0 and 
dn,eif) > 0 for e G JC and f G SC, except when (e,/) = 
(1,1). Following a similar analysis as in the previous case. 


dnAf) 


Bn + 'LeeX D/g S Pejf) C« 
{KT-l)pe{f) 




Note that since £/„.i(l) = 0, there holds 


B„ = Cn[KTpi{l)- Y E w (0 
V keJtteS' J 


Thus, it is straightforward to verify that 


dn,e{f) 


Bn + 'LkeJtr'Lte3rPk{t)Q _ ^ 

KTpeif) 


which matches the expression (fOl l. By the budget constraint 
(i-e-, 'LkejG'Lte^Pk{t)dn,k{t) <B„), we can see that when 
dn.k{t) = 0 for all k G SC and t G SC, user n has a zero 
budget (i.e., B„ = 0). The announced prices cannot be inhnite 
because of the nature of the Stackelberg equilibrium, as we 
discuss later. The following theorem states the necessary and 
sufficient condition for so that the above d„k{tys are 
guaranteed to be feasible. 

Theorem 1: For each user n G ^yC, the demand d„ i^{t) 
given by ([T3l l is feasible if and only if 


Bn > max{fnAjn,2} 


where 


fnA = i;n{KTpk{t)-Y YPkit) 


keAeteS" 


fn,2 = 


CnKT 


V V 1 —c«EE /^^(0 
Lkejxr LteS TT/W) keJG 


Proof: Suppose hrst that B„ > fn,\- Then, 

Bn > CniKTpkf) - E E 's/kGJC,tGSC 

keJtfteS' 


which leads to 

Bn + Cn yikeJG Pkjt) 
KTpkf) 


-Cn>0SkGSC,tGSC 











Thus, the inequality > /„ i implies that the non-negativity 
condition is satisfied. Next suppose that >/«,2- Then, 


Bn> 


CnKT 


'LkeJir'Lte^ 


KTpt(t) 


Thus, 


E L Pkif) 

kejff teS" 


Bn Cn E Pki^) — 
keJXkteS 


Ef" + CnKT 


was a function of the prices) here and show that in the case 
when the equality in (6) is satisfied, the corresponding prices 
constitutes the best response of UC k subject to the prices 
set by the other UCs. 

With the equality in (|6]l and relation (fOT l. there holds 

T,nejk'Bn-\- ^nY,keJ(kY,teS' Pkjt) 

KTpk{t) 

— E Cn + G^jt(0: V f € 


from which we have 


E E 

kej(f te3f 


Bn + CnUkekX^Yite^ Pk{t) 
KTpkit) 


E Ec«>£; 

keX tesr 


This leads to 


E E dn.k{t) > Er 

kejXTteJ' 


With this, the inequality B„ > /„ 2 implies that the minimum 
energy need is satisfied. Combining both conditions, we 
conclude that the condition 


Bn >max{/„j,/„^2} 

guarantees that the maximizing demand in (fOl) is feasible. 
Now suppose that 

Bn + CnUkeJ^^te^ Pk{t) 

KTpk{t) 

It follows that 


- > 0 VA: e f e 


Bn > CniKTpkit) -Y.11 Pkit)) e JT, f e 
keJfteS' 

which implies that > /„ ]. Finally, suppose that 

J ff\ v f^>i +CnllkeJ(fT,te^ Pkit) ^ 

L<k[t) = - KTn.lf-) - 

keJfTteSr keJtteS pk{i) 


> E"" 

— ^n 


Then, 


Y Y Bn + Cn'LkeJf'LteSPkjt) ^ ^min _|_ r 

khfh KTp,{t) 

By re-arranging, we have 


Let Z = Cn ^nd B = Y.ne.A^B„. Then, for each company 

k&Jf, 

B-I-ZE Y.Pk{t)=KTpk{t){Gk{t)+Z), 'it&3r 

keJtteS 

Note that the double summation includes Pk(t) and all the 
other prices. Thus, 

B + Z ^ Y. Pe{h)=KTpk{t){Gk{t)+Z)-pk{t)Z, 

eeXhe.^ (14) 

V t € k G , {e,h) ^ {k,t) 

Note that the equations in (fl4l l can be combined into a linear 
equation 

AP = Y 

where A is a KT x KT matrix whose diagonal entries are 
KT{Gk{t) +Z)—Z, k G GZZ , t G SZ, and off-diagonal entries 
all equal —Z, B is a vector in stacking pk{t), k G .ZP, 
t G 3Z, and Y a vector in whose entries all equal B. 

The following theorem shows that matrix A is invertible 
and the revenue-maximizing prices are positive and unique. 
Theorem 2: The following statements are true. 

1) The matrix A is nonsingular and the prices announced 
by company k G ZZf at time t G SZ are uniquely given 
by 

= G,{t)+Z 

(15) 

2) The above prices are always positive. 

3) The price given by (fTSl l constitutes the best response 
of company k to the prices set by other companies. 


ptnin I ^ T^’T' 

Bn> ^ 1 -C«E LPkit)=fn,2 

LkeJGhteZ KTpi,(t) keJGteJ' 

from which it follows that B„ > max{/„ i,/„2} when the 
demand is feasible. ■ 


B. Leaders-Side Analysis 

Given the prices set by the other companies and subject to 
the power availability constraint (|6]l, each UC (leader) aims 
to determine its most profitable prices. At the leaders level, 
there is a noncooperative game in which each UC chooses 
its optimal prices in response to the prices set by the other 
UCs. The revenue function of company k is an increasing 
function of the consumers’ demands Y.neJ^ ‘^n,k{t). Thus, the 
optimality is reached when the equality in (|6]l holds. We 
apply the solutions derived in the users-side analysis (which 


Proof: 

1) The matrix A can be represented as 


A = 


(KT(Gif)+2) 0 

0 KT(Gi{2)GZ) 


0 KT(Gk(T)+Z)) 
-Z 

(1...1) + i 

\-zJ 


Note that A is invertible (diagonal matrix with nonzero 
diagonal elements). By Sherman-Morrison Formula 
[20], if A is invertible and 1 -|-v^A^'m f 0, then 


(A- 


=A-‘-: 


l+v'k'A-^u 





















Note that 


\+v‘ A^'u = \- 


I Kr(Gi(l)+Z) 

0 


( 1 - 1 ) 


AT(Gi(2)+Z) 


1 


KT(Gk{T)\T)I 


\ (Z\ 

z 

\zj 


= 1 


KT 


E E 


Gk{t) + Z 


Since Gk{t) > 0 and Z > 0, each element in the 
summation is less than 1 and overall value of the 
summation is less than KT, and this clearly leads to 
1 + 7^ 0. Note that 


A-'uv^A-' = 


-Z 

[KTf 


and hence KT < 0. But K>\ and T >\. Thus, this 
is a contradiction. Therefore, we conclude that pk{t) > 
0 Vf e 

3) Given the prices announced by other UCs, suppose 
that a UC k announces a price of = Pk{t) + £ at 
a fixed time f. If £ < 0, the company will decrease its 
price. But, the total demand from all users from UC 
k cannot exceed the power availability (recall that the 
solution in 1) happens when all the available power 
is being sold). Since the revenue at time t is the total 
demand multiplied by the price, UC k does not achieve 
a higher revenue when £ < 0. Now suppose that £ > 0; 
this leads to 

k(t)) ~ k(t)) 


(Gl(lj)+Z)'- 

(Gi(I)+Z)(Gi(2)+Z) 


(Gi(2)+Z)2 


(Gi(l)+Z)(GK(r)+Z) 

1 

(Gi(2)+Z)(G;f(r)+Z) 


V(Gl(l)+Z)(Gy(n+Z) 


(G;f(r)+Z)2 


Thus, 


A^‘ = 


( Kr(Gi'(l)+Z) 
0 


V 


0 

1 

KT{G\{1)^Z] 


{KTf-KT^kexi:,e^G^ 

( 


(Gi{lJ+Z )2 
{Gi{l)+Z)(G,(2)+Z) 


\(G,(l)+Z)(G,f(r)+Z) 


(Gi(2)+Z)2 


0 

0 

kjWPT+zI' 


(Gi(l)+Z)(Gx(r)+Z)\ 

(Gi(2)+Z)(Gx(r)+Z) 


(Gjf(r)+z)2 / 


Since P =A 'T, the price selection is uniquely given 
by 


Pk{t) = 


B 


KT{Gk{t)+Z) 


1 + - 


HkeJpUteS 


Gk(t)+Z 


(16) 

which simplifies to the expression (fTSl ). 

2) Suppose that to the contrary, pkit) < 0, this leads to 


B 


KT{Gk(t)+Z) 


1 + - 




Gk(t)+Z 


KT -Y,kej(r'Lte,Z 


<0 


Gk{t)+Z , 


for some t and k. Note that since 


negative, there holds 


KT{Gk(t)+Z) 


IS non- 


1 + - 


Y,keJ(fT,teS 


Gk{t)+Z 


This implies that 
Z 


z — 
Gk(t)+Z 


<0 


^k{t)+Z 


<-Kr- E Et^ 

V keJPteSr ^k\t) 


= {Pk{t) + £) 


B +ZY,te3k Pkjt) +Zs 
KTipk{t) + e) 


-i.Pk{t) + £)z - Pi:(f) —+ Kpk{t) 


KTpk(t) 


= -Ze- 


KT-1 

KT 


But KT > 1 and Z > 1. Thus, if £ > 0, the UC does 
not achieve a higher revenue. Therefore, for every 
period f, company k does not benefit from deviating 
from (fTSl) . Since this applies to every period, it applies 
for the entire time horizon because of the linearity of 
the revenue function (it is a linear combination of the 
demands multiplied by the prices). 

■ 

In practice, due to production costs and market regulations, 
Pk{t) cannot be outside the range of some lower and upper 
bounds for aW t G and k € as in 

[12]. If pk{t) < then pk{t) is set to /?“"(?), and 

similarly for the upper-bound, if pk{t) > '^hen we 

set pk{t) = P™(0- The expression (ITsT l will still hold for 
the other prices because of the convexity of the problem. 

Remark 1: Using the expression Q, it can be verified that 
the revenue function for UC k is 


KT - Zkejfr l.te^ ^+z , 


KgQn,k{Pkil?—k) — Pk{^) 

teSZ ne-yV 

/UneyfBn + ^nY^keJZ^te.Z Pk{t) 


= E w( 0 (- 

teSZ 


KTpk{t) 


B + ZY,keJpT.teSPkjt) ^ Pk{t) 
teSZ te,Z 

. ^ , K{Y,kejr'Lt€J-Pkjt) -pkjtj) 

K K 


- E c«) 


(17) 


We can see that the revenue function of each company 
depends on the price selections of other companies and thus 
it is appropriate to use the notion of Nash equilibrium. 

Remark 2: Since Z = i^„ is typically 1, the 

value of Z typically equals N. In this case, by ([TST i, we 
observe that for any given Gk{tys, 

Pk{t)iGkit)+Z) = pk{t){Gkit)+N) 
























































which is a constant for all f G ^ and k G Thus, the power 
availability is inversely proportional to the prices. 

Remark 3: Theorem 2 provides a computationally cheap 
expression for the prices. Since pk{t) can be directly com¬ 
puted using (fTSl) . there is no need to numerically compute 
or |A|. This enables us to deal with a large number 
of periods or UCs, without worrying about computational 
complexity. 

C. Existence and Uniqueness of the Stackelberg Equilibrium 

Companies play a noncooperative game at the market level 
and announce their prices to the consumers. As discussed 
above, consumers optimally respond to these prices and the 
response of user n G jV is uniquely given by (fOl l. We assume 
that the participants of the game at the followers level have 
a sufficient budget to participate (derived in Theorem 1). 
Denote the strategy space of user (follower in the game) 
n G jV by and the strategy space of all users by = 

X • • • X Denote the strategy space of UC (a leader in 

the game) k G JT at f G by Note 

that pk(t) € for all f G and k G JU. The strategy space 
of UC k for the entire time horizon is = ■^k,i x • • • x ^kj 
and the strategy space of all companies is .if = .if x • • • x 
.iff. Before stating our main theorem, we need the following 
game-theoretic concepts from [ 6 ]. 

The vector of prices p* G .if constitutes a Nash equilib¬ 
rium for the price selection game at the UCs-level if 

f^gen,/: {phV-k)>Ugen,k{Pk,P-k*), ypk^-^k 

For given price selections {pi,...,pk) G .if x • • • x .iff, the 
optimal response from all users is 

d*(p) = K(p),(i|(p),...,4(p)} 

where for each n G yU, d*(p) is the unique maximizer for 
Uusei,n{dn,p) over dn G ,^n- For the game considered here, 
the Stackelberg equilibrium is defined as (d*(p),p*). 

In general, in Stackelberg games, the response from the 
followers has to be unique for the equilibrium to be well 
defined [ 6 ]. In the game here, due to market competition, 
leaders aim to choose their prices in the most profitable 
way while taking into account what other leaders are doing. 
We capture the competition on the leaders’ level through a 
Nash game. We note that in the parlance of dynamic game 
theory [ 6 ], we are dealing here with open-loop information 
structures, with the corresponding equilibria at the utilities 
level being open-loop Nash equilibrium. Therefore, this is a 
one-shot game at which all the prices for the all periods are 
announced at the beginning of the game, and the followers 
respond to these prices by solving their local optimization 
problems. We have the following theorem: 

Theorem 3: The following statements are true. 

1) There exists a unique (open-loop) Nash equilibrium for 
the noncooperative game at the leaders’ level. 

2) There exists a unique (open-loop) Stackelberg equilib¬ 
rium. 


3) The maximizing demands given by (fTsT i and the 
revenue-maximizing prices given in Theorem 2 con¬ 
stitute the (open-loop) Stackelberg equilibrium for the 
demand response management game. 

Proof: 

1) Note that the set ^k,t '■= \pT”i^)^P’k“^i^)\ G R is com¬ 
pact (closed and bounded) 'it € ^,k £ JU . Thus, = 
££\ X • • • X .iff is compact subset of Furthermore, 

d^lJsen,k{Pk{t) V-,f) VfG,i7,kGjr 

dPkil) 

since the revenue function is convex and concave 
in pk{t). Hence, there exists a Nash equilibrium for 
the noncooperative game at the leaders level [6]. By 
Theorem 2, pk{t) that constitutes the best response of 
company k G to prices set by other companies is 
uniquely given by (flST l. Thus, the Nash equilibrium is 
unique. 

2) From 1), a unique Nash equilibrium exists at which the 
maximizing prices are announced to the consumers. 
Since the maximizing demands were uniquely given 
by (foi l, then there exists a unique Stackelberg equi¬ 
librium. 

3) Immediately follows from Theorem 2 and parts l)-2) 

■ 

IV. Numerical Results 

We conducted numerical computations capturing different 
scenarios. In the first case, we set T = 1 and compute the 
results with the same parameters values as in the single¬ 
period case [12]. Here, forn G JV, we have = 1 and 7 „ = 1. 
Additionally, K = 3>,N = S,B\ varies from 2 to 42, = 10, 

B 3 = 15, B 4 = 20, and B 5 = 25. The power availabilities are 
Gi(l) = 10, ^ 2 ( 1 ) = 15, and G 3 (l) = 20. Figure 1 shows 
that the results matches the single-period case given in [ 12 ]. 
This is expected since the multi-period Stackelberg game is 
a generalization of the single-period one. Now, we let T = A, 
an interpretation for which can be the following: morning, 
afternoon, evening, and late night. The budgets of consumers 
are kept the same, and as before, B\ still varies from 2 to 
42. The total power availability for each company is also 
kept at the same level, but distributed across the 4 periods 
as follows: 25%, 40%, 25%, and 10%. Figure 2 shows that 
with the same total power availability for each company, 
and without increasing any of the consumers’ budgets, the 
utilities for users increase significantly (almost doubles for 
user 1). The total demands for the users do not change, since 
they match the power availability. The trend for the revenues 
is similar to the single-period case. One key observation is 
that the multi-period scheme provides more incentives for 
consumers’ participation (can be shown analytically), which 
is quite important and a key issue [2]. Finally, we now 
increase the number of users and study the behaviors with 
a varying T. Here, T varies from 1 to 50, N = 50, K = I, 
Gi (f) = 'i t £ (so, the power availability is equally 
distributed among all periods). Bi_io = 5, Bii -20 = 10, 
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Fig. 1. Computations with 5 users, T — and varying B\ 


^21-30 = 15, B 31-40 = 20, and B 41-50 = 25. Figure 3 shows 
that the previous observation again holds, and increasing the 
number of the periods provides more incentives for users’ 
participation without affecting the revenue of the company. 

V. Conclusions and Future Directions 

In this paper, we have studied the multi-period DRM prob¬ 
lem through game-theoretic methods. In particular, we have 
developed a Stackelberg game to capture the interactions 
between UCs and energy consumers in which UCs are the 
leaders and the users are the followers of the game. The 
UCs play a noncooperative game, which was shown to have 
a unique equilibrium at which companies maximize their 
revenues in response to price decisions made by the other 
companies. Then, the users optimally respond to the price 
selection made by UCs, by choosing the utility maximizing 
demand. The overall hierarchal interaction admits a unique 
Stackelberg equilibrium. Furthermore, closed-form solutions 
have been derived, and numerical results have shown that 
the multi-period scheme provides more incentives for the 
participation of energy consumers. 

There are numerous opportunities for future work. Incor¬ 
porating the economic dispatch problem at the UC level is 
one possible direction. Another direction is to study energy 
scheduling and storage at the consumers level. Moreover, 
as this paper has focused on shiftable energy need at the 


consumer level, another opportunity is to study the addition 
of some period-specihc energy need constraints. 
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